Learning object, grasping and manipulation activities using hierarchical HMMs

This article presents a probabilistic algorithm for representing and learning complex manipulation activities performed by humans in everyday life. The work builds on the multi-level Hierarchical Hidden Markov Model (HHMM) framework which allows decomposition of longer-term complex manipulation activities into layers of abstraction whereby the building blocks can be represented by simpler action modules called action primitives. This way, human task knowledge can be synthesised in a compact, effective representation suitable, for instance, to be subsequently transferred to a robot for imitation. The main contribution is the use of a robust framework capable of dealing with the uncertainty or incomplete data, and the ability to represent behaviours at multiple levels of abstraction for enhanced taks generalisation. Activity data from 3D video sequencing of human manipulation of different objects handled in everyday life is used for evaluation. A comparison with a mixed generative-discriminative hybrid model HHMM/SVM (Support Vector Machine) is also presented to add rigour in highlighting the benefit of the proposed approach against comparable state of the art techniques. Mitesh Patel, Jaime Valls Miro and Gamini Dissanayake Faculty of Engineering and IT, University of Technology Sydney (UTS), 15 Broadway, Ultimo, NSW-2007, Australia Tel.: +612-95143146 Fax: +612-95142655 E-mail: [email protected], [email protected], [email protected] Danica Kragic, Carl Henrik Ek Members of the Computer Vision & Active Perception Lab., Centre for Autonomous Systems, School of Computer Science and Communication, The Royal Institute of Technology (KTH), Stockholm, Sweden E-mail: [email protected], [email protected]